foundation model

A foundation model is anymodel that is trained on broad data (generally using self-supervision at scale) that can be adapted(e.g., fine-tuned) to a wide range of downstream tasks; current examples include BERT [Devlin et al .2019], GPT-3 [Brown et al . 2020], and CLIP [Radford et al . 2021].

On the Opportunities and Risks of Foundation ModelsOn the Opportunities and Risks of Foundation Models 2023

Many of these models are described as foundation models. This term, coined in 2021 by researchers at the Stanford Institute for Human-Centered Artificial Intelligence, broadly describes AI models that acquire the capacity to perform well across a range of different problems through a process of ‘pretraining’ on enormous unannotated datasets.

Foundation models build on ChatGPT tech to learn the fundamental language of biology - Nature Biotechnology Michael Eisenstein 2024